Add runtime migration creation and application #37415

thromel · 2025-12-23T07:41:45Z

Summary

Implements #37342: Allow creating and applying migrations at runtime without recompiling.

This adds support for creating and applying migrations at runtime using Roslyn compilation, enabling scenarios like .NET Aspire and containerized applications where recompilation isn't possible.

CLI Usage

# Standard update (existing behavior)
dotnet ef database update [migration]

# Create and apply a new migration in one step
dotnet ef database update MigrationName --add [--output-dir <DIR>] [--namespace <NS>] [--json]

The -o/--output-dir, -n/--namespace, and --json options require --add to be specified.

PowerShell Usage

# Standard update (existing behavior)
Update-Database [-Migration <migration>]

# Create and apply a new migration in one step
Update-Database -Migration MigrationName -Add [-OutputDir <DIR>] [-Namespace <NS>]

Architecture

Component	Purpose
`IMigrationCompiler` / `CSharpMigrationCompiler`	Internal: Roslyn-based compilation of scaffolded migrations
`IMigrationsAssembly.AddMigrations(Assembly)`	Registers dynamically compiled migrations
`MigrationsOperations.AddAndApplyMigration()`	Orchestrates scaffold → compile → register → apply workflow

Design Decisions

Extends existing services: Uses IMigrationsScaffolder for scaffolding and IMigrator for applying, adding only the new IMigrationCompiler service
AddMigrations(Assembly): Extended IMigrationsAssembly interface to accept additional assemblies containing runtime-compiled migrations
Always persists to disk: Like AddMigration, files are always saved to enable source control and future recompilation
No pending changes behavior: If no model changes are detected, applies any existing pending migrations without creating a new one
Internal compiler API: IMigrationCompiler and CSharpMigrationCompiler are in the .Internal namespace as they require design work for public API
Error handling with cleanup: If compilation or migration application fails, saved migration files are cleaned up to prevent orphans
Thread safety: MigrationsAssembly uses locking to protect against race conditions when adding migrations concurrently

Workflow

User runs: dotnet ef database update InitialCreate --add
    │
    ▼
MigrationsOperations.AddAndApplyMigration()
    │
    ├─► Check for pending model changes
    │       └─► If none: apply existing migrations, return
    │
    ├─► IMigrationsScaffolder.ScaffoldMigration() - Generate code
    │
    ├─► try {
    │       ├─► IMigrationsScaffolder.Save() - Write files to disk
    │       ├─► IMigrationCompiler.CompileMigration() - Roslyn compile
    │       ├─► IMigrationsAssembly.AddMigrations() - Register migration
    │       └─► IMigrator.Migrate() - Apply to database
    │   } catch {
    │       └─► Clean up saved files on failure
    │   }
    │
    └─► Return migration files

Robustness Features

Exception handling with cleanup: AddAndApplyMigration wraps the save-compile-register-apply chain in try-catch, deleting saved files on failure to prevent orphans
Context disposal on validation failure: PrepareForMigration ensures the DbContext is disposed if validation or service building fails
Thread-safe migration registration: MigrationsAssembly uses locking to protect shared state (migrations dictionary, model snapshot, additional assemblies list)

Limitations

Requires dynamic code generation (incompatible with NativeAOT) - marked with [RequiresDynamicCode]
C# only (no VB.NET/F# support)

Test plan

Unit tests for CSharpMigrationCompiler
Unit tests for MigrationsOperations.AddAndApplyMigration
Integration tests in RuntimeMigrationTestBase (SQLite and SQL Server implementations)
Tests for validation (empty name, invalid characters)
Tests for RemoveMigration with dynamically created migrations
All existing EFCore.Design.Tests pass
All existing EFCore.Relational.Tests pass

Fixes #37342

test/EFCore.SqlServer.FunctionalTests/Migrations/RuntimeMigrationSqlServerTest.cs

src/ef/Commands/DatabaseUpdateCommand.Configure.cs

src/ef/IOperationExecutor.cs

src/EFCore.Design/Design/Internal/MigrationsOperations.cs

src/EFCore.Design/Design/DesignTimeServiceCollectionExtensions.cs

src/EFCore.Design/Migrations/Design/IDynamicMigrationsAssembly.cs

src/ef/Commands/DatabaseUpdateCommand.cs

src/EFCore.Relational/Extensions/RelationalDatabaseFacadeExtensions.cs

src/EFCore.Design/Design/Internal/MigrationsOperations.cs

src/EFCore.Design/Migrations/Design/RuntimeMigrationService.cs

test/EFCore.Design.Tests/Design/Internal/MigrationsOperationsTest.cs

src/ef/Commands/DatabaseUpdateCommand.cs

src/EFCore.Design/Design/Internal/MigrationsOperations.cs

src/ef/Commands/DatabaseUpdateCommand.cs

- Implement IAsyncLifetime and call Fixture.ReseedAsync() in InitializeAsync instead of manually calling CleanDatabase in each test - Use context.Database.OpenConnection/CloseConnection instead of direct connection.Open/Close calls - Move database cleanup logic to fixture's CleanAsync override - Add GetTableNamesAsync to fixtures for async cleanup

- Simplify CSharpMigrationCompiler.GetMetadataReferences to use cached references plus context assembly, removing explicit Assembly.Load calls - Remove duplicate name validation from AddMigration/AddAndApplyMigration since PrepareForMigration already validates - Remove UsePooling override (controls DbContext pooling, not connection pooling)

src/EFCore.Design/Design/Internal/MigrationsOperations.cs

Copilot

Pull request overview

This PR implements runtime migration creation and application to support scenarios like .NET Aspire and containerized applications where recompiling is not possible. It extends the existing dotnet ef database update and Update-Database commands with a new --add option that scaffolds, compiles (using Roslyn), registers, and applies a migration in one atomic operation.

Changes:

Adds IMigrationCompiler interface and CSharpMigrationCompiler implementation for runtime Roslyn-based compilation of scaffolded migrations
Extends IMigrationsAssembly with AddMigrations(Assembly) method to register dynamically compiled migrations
Adds AddAndApplyMigration operation to MigrationsOperations that orchestrates the scaffold → compile → register → apply workflow
Updates CLI and PowerShell commands to support --add, --output-dir, --namespace, and --json options with appropriate validation

Reviewed changes

Copilot reviewed 26 out of 29 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
`src/EFCore.Design/Migrations/Design/IMigrationCompiler.cs`	New internal interface for runtime migration compilation
`src/EFCore.Design/Migrations/Design/CSharpMigrationCompiler.cs`	Roslyn-based implementation with assembly reference caching
`src/EFCore.Relational/Migrations/IMigrationsAssembly.cs`	Adds AddMigrations method to public interface
`src/EFCore.Relational/Migrations/Internal/MigrationsAssembly.cs`	Implements dynamic migration registration with thread-safety concerns
`src/EFCore.Design/Design/Internal/MigrationsOperations.cs`	Core AddAndApplyMigration operation with error handling
`src/EFCore.Design/Design/OperationExecutor.cs`	Operation executor for AddAndApplyMigration command
`src/ef/Commands/DatabaseUpdateCommand*.cs`	CLI command extensions with validation logic
`src/EFCore.Tools/tools/EntityFrameworkCore.psm1`	PowerShell Update-Database function enhancements
`test/EFCore.Relational.Specification.Tests/RuntimeMigrationTestBase.cs`	Comprehensive test base with 20+ test scenarios
`test/EFCore..FunctionalTests/RuntimeMigrationTest.cs`	Provider-specific test implementations
Resource files (`.resx`, `.Designer.cs`)	New localized strings for errors and messages

Files not reviewed (3)

src/EFCore.Design/Properties/DesignStrings.Designer.cs: Language not supported
src/dotnet-ef/Properties/Resources.Designer.cs: Language not supported
src/ef/Properties/Resources.Designer.cs: Language not supported

src/EFCore.Relational/Migrations/Internal/MigrationsAssembly.cs

src/EFCore.Design/Design/Internal/MigrationsOperations.cs

src/ef/Commands/DatabaseUpdateCommand.Configure.cs

src/EFCore.Design/Design/DesignTimeServiceCollectionExtensions.cs

test/EFCore.Relational.Specification.Tests/RuntimeMigrationTestBase.cs

src/EFCore.Tools/tools/EntityFrameworkCore.psm1

src/EFCore.Design/Design/Internal/MigrationsOperations.cs

…ions Per reviewer feedback, added a `createTables` parameter (default true) to the test store clean methods. Runtime migration tests use `createTables: false` to get an empty database without tables, allowing migrations to create them. Changes: - Add `bool createTables = true` to RelationalDatabaseCleaner.Clean() - Propagate parameter through SqliteDatabaseCleaner, SqlServerDatabaseCleaner - Add parameter to EnsureClean extension methods - Add parameter to TestStore.CleanAsync and provider implementations - Simplify RuntimeMigrationTestBase to use TestStore.CleanAsync directly - Add UsePooling => false to fixture (pooled contexts retain migration assemblies) - Remove custom CleanAsync overrides from runtime migration fixtures

- Add validation that --json requires --add in database update command - Restore original service registration order in DesignTimeServiceCollectionExtensions - Add PowerShell validation for -OutputDir/-Namespace requiring -Add - Add comment documenting empty MigrationFiles JSON behavior

Instead of adding a new custom resource JsonRequiresAdd, reuse the existing MissingConditionalOption resource which provides the same functionality. This avoids issues with T4 template regeneration in CI.

The AddAndApplyMigration tests require a database connection because Migrator.Migrate() calls _connection.Open(). Without a valid connection string, the tests fail in CI. Adding "Data Source=:memory:" provides an in-memory SQLite database that allows the migration operations to complete successfully.

The Migrator.Migrate method opens and closes the connection multiple times during migration (for CreateIfNotExists and MigrateImplementation). With Data Source=:memory:, the SQLite database is destroyed when the connection closes, which causes the migration history table to be lost between steps. Using an externally opened connection ensures EF Core won't close it during migration, keeping the in-memory database alive throughout the operation.

AndriySvyryd · 2026-01-12T22:34:50Z

src/ef/Commands/DatabaseUpdateCommand.Configure.cs

+            if (_outputDir!.HasValue())
+            {
+                throw new CommandException(Resources.OutputDirRequiresAdd);
+            }
+
+            if (_namespace!.HasValue())
+            {
+                throw new CommandException(Resources.NamespaceRequiresAdd);


Use Resources.MissingConditionalOption for these exceptions as well instead of adding new string resources

AndriySvyryd · 2026-01-12T23:46:42Z

test/EFCore.Design.Tests/Design/Internal/MigrationsOperationsTest.cs

+    [ConditionalFact]
+    public void AddMigration_throws_when_name_is_empty()
+    {
+        var assembly = MockAssembly.Create(typeof(TestContext));


Use AssemblyTestContext in these tests

AndriySvyryd · 2026-01-13T00:09:06Z

src/ef/Commands/DatabaseUpdateCommand.cs

+
+            if (_json!.HasValue())
+            {
+                ReportJson(files);


If files is empty or just contains null values don't report anything.

AndriySvyryd · 2026-01-13T00:15:30Z

src/EFCore.Design/Design/Internal/MigrationsOperations.cs

+            throw new OperationException(
+                DesignStrings.AddAndApplyMigrationFailed(name, ex.Message), ex);


You don't need to wrap this exception, so remove the try/catch

AndriySvyryd · 2026-01-13T00:30:45Z

src/EFCore.Design/Design/Internal/MigrationsOperations.cs

+        var migration =
+            string.IsNullOrEmpty(@namespace)
+                ? scaffolder.ScaffoldMigration(name, _rootNamespace ?? string.Empty, subNamespace, _language, dryRun: true)
+                : scaffolder.ScaffoldMigration(name, null, @namespace, _language, dryRun: true);
+
+        MigrationFiles? files = null;
+        try
+        {
+            files = scaffolder.Save(_projectDir, migration, resolvedOutputDir, dryRun: false);


Move this, as well as scaffolder declaration to a new method that can also be called by AddMigration, make sure to preserve the code comment about #18950.

outputDir and subNamespace calculations can be moved from PrepareForMigration to this new shared method.

AndriySvyryd · 2026-01-13T00:45:41Z

src/EFCore.Design/Migrations/Design/CSharpMigrationCompiler.cs

+        if (_cachedReferences != null)
+        {
+            return _cachedReferences;
+        }
+
+        lock (_referenceLock)
+        {
+            if (_cachedReferences != null)
+            {
+                return _cachedReferences;
+            }
+
+            var references = new List<MetadataReference>();
+
+            // Add references from all loaded assemblies (except dynamic/in-memory ones)
+            foreach (var assembly in AppDomain.CurrentDomain.GetAssemblies())
+            {
+                AddAssemblyReference(references, assembly);
+            }
+
+            _cachedReferences = references;
+            return _cachedReferences;
+        }


Use NonCapturingLazyInitializer.EnsureInitialized to avoid using a lock

AndriySvyryd · 2026-01-13T01:02:56Z

src/EFCore.Design/Migrations/Design/CSharpMigrationCompiler.cs

+    /// <returns>The list of metadata references.</returns>
+    protected virtual IReadOnlyList<MetadataReference> GetMetadataReferences(
+        Type contextType,
+        IEnumerable<Assembly>? additionalReferences)


It seems that additionalReferences is always null, so it can be removed.

AndriySvyryd · 2026-01-13T01:06:32Z

src/EFCore.Design/Migrations/Design/CSharpMigrationCompiler.cs

+        var allReferences = new List<MetadataReference>(baseReferences);
+
+        // Add the context's assembly (in case it wasn't loaded when cache was built)
+        AddAssemblyReference(allReferences, contextType.Assembly);


I can't see a situation where the context assembly wouldn't be in the cache, so you can remove this call and this whole method can then be inlined

AndriySvyryd · 2026-01-13T01:10:39Z

src/EFCore.Design/Migrations/Design/IMigrationCompiler.cs

+    /// <summary>
+    ///     Compiles scaffolded migration source code into an in-memory assembly.
+    /// </summary>
+    /// <param name="scaffoldedMigration">The scaffolded migration containing C# source code.</param>
+    /// <param name="contextType">The type of the <see cref="DbContext" /> for which the migration was created.</param>
+    /// <param name="references">Additional assembly references to include in compilation, if any.</param>
+    /// <returns>An <see cref="Assembly" /> containing the compiled migration and model snapshot.</returns>
+    /// <exception cref="InvalidOperationException">Thrown when compilation fails.</exception>


Suggested change

/// <summary>

/// Compiles scaffolded migration source code into an in-memory assembly.

/// </summary>

/// <param name="scaffoldedMigration">The scaffolded migration containing C# source code.</param>

/// <param name="contextType">The type of the <see cref="DbContext" /> for which the migration was created.</param>

/// <param name="references">Additional assembly references to include in compilation, if any.</param>

/// <returns>An <see cref="Assembly" /> containing the compiled migration and model snapshot.</returns>

/// <exception cref="InvalidOperationException">Thrown when compilation fails.</exception>

/// <summary>

/// This is an internal API that supports the Entity Framework Core infrastructure and not subject to

/// the same compatibility standards as public APIs. It may be changed or removed without notice in

/// any release. You should only use it directly in your code with extreme caution and knowing that

/// doing so can result in application failures when updating to a new Entity Framework Core release.

/// </summary>

AndriySvyryd · 2026-01-13T01:13:29Z

src/EFCore.Relational/Migrations/Internal/MigrationsAssembly.cs

+    {
+        get
+        {
+            lock (_lock)


Use NonCapturingLazyInitializer.EnsureInitialized here instead of lock

AndriySvyryd · 2026-01-13T01:30:07Z

test/EFCore.Design.Tests/Migrations/Design/CSharpMigrationCompilerTest.cs

+    }
+
+    [ConditionalFact]
+    public void CompileMigration_throws_on_very_long_migration_name()


This test is just testing the compiler; remove.

AndriySvyryd · 2026-01-13T01:30:54Z

test/EFCore.Design.Tests/Migrations/Design/CSharpMigrationCompilerTest.cs

+    }
+
+    [ConditionalFact]
+    public void CompileMigration_throws_on_invalid_code()


This test is just testing the compiler; remove.

AndriySvyryd · 2026-01-13T01:51:44Z

test/EFCore.Relational.Specification.Tests/RuntimeMigrationTestBase.cs

+    }
+
+    [ConditionalFact]
+    public void Can_compile_migration()


Try to refactor some of these tests to create a MigrationsOperations with a TestOperationReporter and call the extracted methods on MigrationsOperations mentioned in the comments above to reduce code duplication in these tests (the extracted methods would need to be public)

thromel force-pushed the feature/runtime-migrations branch from a15611a to 9a35a9b Compare December 23, 2025 20:45

AndriySvyryd added the community-contribution label Dec 23, 2025

AndriySvyryd self-assigned this Dec 23, 2025

thromel marked this pull request as ready for review December 24, 2025 06:22

thromel requested a review from a team as a code owner December 24, 2025 06:22

thromel marked this pull request as draft December 25, 2025 02:03

thromel force-pushed the feature/runtime-migrations branch from 62018f1 to 5c41f2f Compare December 25, 2025 03:51

thromel marked this pull request as ready for review December 25, 2025 08:13

thromel marked this pull request as draft December 25, 2025 16:43

thromel marked this pull request as ready for review December 25, 2025 21:07

AndriySvyryd reviewed Dec 30, 2025

View reviewed changes

test/EFCore.SqlServer.FunctionalTests/Migrations/RuntimeMigrationSqlServerTest.cs Outdated Show resolved Hide resolved

AndriySvyryd reviewed Dec 30, 2025

View reviewed changes

src/ef/Commands/DatabaseUpdateCommand.Configure.cs Show resolved Hide resolved

AndriySvyryd reviewed Dec 30, 2025

View reviewed changes

src/ef/IOperationExecutor.cs Outdated Show resolved Hide resolved

AndriySvyryd reviewed Dec 30, 2025

View reviewed changes

src/EFCore.Design/Design/Internal/MigrationsOperations.cs Show resolved Hide resolved

AndriySvyryd reviewed Dec 30, 2025

View reviewed changes

src/EFCore.Design/Design/Internal/MigrationsOperations.cs Show resolved Hide resolved

AndriySvyryd reviewed Dec 31, 2025

View reviewed changes

src/EFCore.Design/Design/DesignTimeServiceCollectionExtensions.cs Outdated Show resolved Hide resolved

AndriySvyryd reviewed Dec 31, 2025

View reviewed changes

src/EFCore.Design/Migrations/Design/IDynamicMigrationsAssembly.cs Outdated Show resolved Hide resolved

AndriySvyryd reviewed Dec 31, 2025

View reviewed changes

src/EFCore.Design/Migrations/Design/IDynamicMigrationsAssembly.cs Outdated Show resolved Hide resolved

thromel requested a review from AndriySvyryd December 31, 2025 05:21

AndriySvyryd reviewed Jan 2, 2026

View reviewed changes

src/ef/Commands/DatabaseUpdateCommand.cs Outdated Show resolved Hide resolved

AndriySvyryd reviewed Jan 2, 2026

View reviewed changes

src/EFCore.Relational/Extensions/RelationalDatabaseFacadeExtensions.cs Outdated Show resolved Hide resolved

AndriySvyryd reviewed Jan 2, 2026

View reviewed changes

src/EFCore.Design/Design/Internal/MigrationsOperations.cs Outdated Show resolved Hide resolved

AndriySvyryd reviewed Jan 2, 2026

View reviewed changes

src/EFCore.Design/Design/Internal/MigrationsOperations.cs Outdated Show resolved Hide resolved

AndriySvyryd reviewed Jan 2, 2026

View reviewed changes

src/EFCore.Design/Migrations/Design/RuntimeMigrationService.cs Outdated Show resolved Hide resolved

AndriySvyryd reviewed Jan 2, 2026

View reviewed changes

src/EFCore.Design/Migrations/Design/RuntimeMigrationService.cs Outdated Show resolved Hide resolved

This comment was marked as outdated.

Sign in to view

AndriySvyryd reviewed Jan 6, 2026

View reviewed changes

test/EFCore.Design.Tests/Design/Internal/MigrationsOperationsTest.cs Outdated Show resolved Hide resolved

AndriySvyryd reviewed Jan 6, 2026

View reviewed changes

src/ef/Commands/DatabaseUpdateCommand.cs Outdated Show resolved Hide resolved

AndriySvyryd reviewed Jan 6, 2026

View reviewed changes

src/EFCore.Design/Design/Internal/MigrationsOperations.cs Outdated Show resolved Hide resolved

AndriySvyryd reviewed Jan 6, 2026

View reviewed changes

src/ef/Commands/DatabaseUpdateCommand.cs Outdated Show resolved Hide resolved

This comment was marked as resolved.

Sign in to view

AndriySvyryd reviewed Jan 9, 2026

View reviewed changes

src/EFCore.Design/Design/Internal/MigrationsOperations.cs Show resolved Hide resolved

AndriySvyryd requested a review from Copilot January 11, 2026 03:37

Copilot started reviewing on behalf of AndriySvyryd January 11, 2026 03:37 View session

Copilot AI reviewed Jan 11, 2026

View reviewed changes

thromel force-pushed the feature/runtime-migrations branch from 2e06ca9 to 72a3c0f Compare January 11, 2026 07:43

thromel added 4 commits January 11, 2026 13:53

Use existing MissingConditionalOption resource for --json validation

9a62a7b

Instead of adding a new custom resource JsonRequiresAdd, reuse the existing MissingConditionalOption resource which provides the same functionality. This avoids issues with T4 template regeneration in CI.

AndriySvyryd reviewed Jan 12, 2026

View reviewed changes

AndriySvyryd reviewed Jan 13, 2026

View reviewed changes

roji force-pushed the main branch 2 times, most recently from 249ae47 to 6b86657 Compare January 13, 2026 17:46

thromel added 2 commits January 14, 2026 08:49

Use AssemblyTestContext in migration name tests

4193835

Address runtime migrations review feedback

39498fa

		throw new OperationException(
		DesignStrings.AddAndApplyMigrationFailed(name, ex.Message), ex);

Add runtime migration creation and application #37415

Are you sure you want to change the base?

Add runtime migration creation and application #37415

Conversation

thromel commented Dec 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

CLI Usage

PowerShell Usage

Architecture

Design Decisions

Workflow

Robustness Features

Limitations

Test plan

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment was marked as outdated.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment was marked as resolved.

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AndriySvyryd Jan 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

thromel commented Dec 23, 2025 •

edited

Loading

AndriySvyryd Jan 13, 2026 •

edited

Loading